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What is claimed is: 



1 1 . A method of monitoring language understanding of a user's input 

2 communication in an automated dialog system, comprising: 

3 determining whether a probability of understanding the user's input 

4 communication exceeds a first threshold, wherein if the first threshold is 

5 exceeded, further dialog is conducted with the user. 

1 2. The method of claim 1 , wherein if the first threshold is not 

2 exceeded, the user is routed to a human for assistance. 

1 3. The method of claim 1 , further comprising: 

2 determining whether a probability of understanding the user's input 

3 communication exceeds a second threshold, the second threshold being greater 

4 than the first threshold, wherein if the second threshold is exceeded, further 

5 dialog is conducted with the user using a current dialog strategy. 

1 4. The method of claim 3, wherein if the second threshold is not 

2 exceeded, further dialog is conducted with the user using an adapted dialog 

3 strategy. 

1 5. The method of claim 4, wherein the adapted dialog strategy 

2 includes one of prompting the user with choices and prompting the user to 

3 confimi the recognition and understanding data. 

1 6. The method of claim 1 , wherein the user's input communication 

2 includes at least one of verbal and nonverbal communications. 

1 7. The method of claim 6, wherein the nonverbal communications 

2 include at least one of gestures, body movements, head movements, non- 

3 responses, text, keyboard entries, keypad entries, mouse clicks, DTMF codes, 

4 pointers, stylus, cable set-top box entries, graphical user interface entries, and 

5 touchscreen entries. 

1 8. The method of claim 1 , wherein the method is used for customer 

2 care purposes. 

1 9. The method of claim 1 , wherein the probability is determined using 

2 recognition and understanding data derived from the user's input communication. 
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1 10. The method of claim 1 , wherein the probability is determined using 

2 training data stored in a training database, the training data including at least one 

3 of classification models and extracted features. 

1 11. The method of claim 10, wherein the extracted features are derived 

2 from recognition, understanding and dialog data. 

1 12, The method of claim 1 , further comprising: 

2 storing a first dialog exchange in a dialog history database, wherein 

3 the first dialog exchange includes a first automated dialog output and the user's 

4 first input communication and the further dialog conducted with the user results in 

5 a second dialog exchange, wherein the second dialog exchange includes a 

6 second dialog output and the user's second input communication; and 

7 determining whether the probability of understanding exceeds the 

8 first threshold using the first dialog exchange and the second dialog exchange. 

1 13. The method of claim 12, wherein the method is recursive in that the 

2 determining step determines whether the probability of exceeds the first 

3 threshold using the each of the dialog exchanges conducted. 

1 14. The method of claim 1 , further comprising: 

2 receiving the user's input communication; 

3 recognizing portions of the user's input communication; and 

4 providing an input to a language understanding monitor based on 

5 applying a confidence function to the recognized portions of the user's input 

6 communication. 

1 15. A language understanding monitoring system that operates in an 

2 automated dialog system, comprising: 

3 a dialog manager that output dialog to the user; 

4 a language understanding monitor that determines whether a 

5 probability of understanding the user's input communication exceeds a first 

6 threshold, wherein if the first threshold is exceeded, the language understanding 

7 monitor prompts the dialog manager to conduct further dialog with the user. 
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1 16, The system of claim 15, wherein if the first threshold is not 

2 exceeded, the language understanding monitor prompts the dialog manager to 

3 route the user to a human for assistance. 

1 17. The system of claim 15, wherein the language understanding 



2 monitor determines whether a probability of understanding the user's input 

3 communication exceeds a second threshold, the second threshold being greater 

4 than the first threshold, and if the second threshold is exceeded, the language 

5 understanding monitor prompts the dialog manager to conduct further dialog with 

6 the user using a current dialog strategy. 



1 18. The system of claim 17, wherein if the second threshold is not 

2 exceeded, the language understanding monitor prompts the dialog manager to 

3 conduct further dialog with the user using an adapted dialog strategy. 

1 19. The system of claim 18, wherein the adapted dialog strategy 

2 includes one of prompting the user with choices and prompting the user to 

3 confirm the recognition and understanding data. 

1 20. The system of claim 15, wherein the user's input communication 

2 includes at least one of verbal and nonverbal communications. 

1 21 . The system of claim 15, wherein the system is used for customer 

2 care purposes. 

1 22. The system of claim 15, wherein the language understanding 

2 monitor determines the probability using recognition data provided by a 

3 recognizer and understanding data provided by a language understanding unit, 

4 and the recognition and understanding data is derived from the user's input 

5 communication. 

1 23. The system of claim 15, further comprising a training database for 



2 storing training data for language understanding, wherein the language 

3 understanding monitor determines the probability using the training data stored in 

4 the training database, the training data including at least one of classification 

5 models and extracted features. 

1 24. The system of claim 23, wherein the extracted features are derived 

2 from recognition, understanding and dialog data. 
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1 25. The system of claim 15, further comprising: 

2 a dialog history database that stores a first dialog exchange, 

3 wherein the first dialog exchange includes a first automated dialog output and the 

4 user's first input communication, and the further dialog conducted with the user 

5 results in a second dialog exchange, the second dialog exchange including a 

6 second dialog output and the user's second input communication, and the 

7 language understanding monitor determining whether the probability of 

8 understanding exceeds the first threshold using the first dialog exchange and the 

9 second dialog exchange. 



1 26. The system of claim 25, wherein the language understanding 

2 monitor determines whether the probability of understanding exceeds the first 

3 threshold using the each of the dialog exchanges conducted. 

1 27. The system of claim 15, further comprising: 

2 a recognizer that recognizes the user's input communication; and 

3 a language understanding unit that applies a confidence function to 

4 the recognized portions of the user's input communication and provides an input 

5 to the language understanding monitor. 

1 28. A method of monitoring language understanding of a user's input 

2 communication in an automated dialog system, comprising: 

3 determining whether a probability of understanding the user's input 

4 communication exceeds a first threshold, wherein if the first threshold is 

5 exceeded, further dialog is conducted with the user, otherwise, the user is routed 

6 to a human for assistance; 

7 determining whether a probability of understanding the user's input 

8 communication exceeds a second threshold, the second threshold being greater 

9 than the first threshold, wherein if the second threshold is exceeded, further 

10 dialog is conducted with the user using a current dialog strategy, otherwise 

1 1 further dialog is conducted with the user using an adapted dialog strategy. 
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